Selected past Research

I want to develop new compilers, architectures, and algorithms that enable graphics and imaging applications orders of magnitude more rich than any we have today. For example, replacing video with real-time 4D light elds, synthesizing trillions of voxels to rapidly print large 3D objects with micron level details, rendering interactive worlds indistinguishable from reality, and building machines which pervasively understand the visual world all require orders of magnitude more computational power than we have today. With current technology, Lytro’s light eld camera can only render thumbnail-resolution previews on the device, and cannot capture video, while processing the light eld it captures to produce a single sub-megapixel nal image takes up to 30 seconds on a desktop PC. Orders of magnitude more energy efficiency will let us leverage cheap, high data rate cameras for everything from human-computer interaction to search, and move powerful rendering and image processing into our glasses where it will transform how we see, think, remember, and are entertained. Always-on cameras will be able to unintrusively monitor our vital signs and health throughout our lives. Super cially, the data parallelism inherent in graphics and imaging computations would seem to make them easy to scale on future hardware. Real graphics and imaging computations, however, have complex dependencies, and are limited by locality (the distance over which data has to move, e.g. from nearby caches or far away main memory) and synchronization. Increasingly, the cost of communication—both within a chip and over a network—dominates computation and power consumption, and limits the gains realized from shrinking transistors. Relative to performing an ALU operation on 32 bits of data, moving that data a few millimeters across a chip requires an order of magnitude more energy, moving it to or from off-chip DRAM requires four orders of magnitudemore, and sending it over a cellular radio at least six orders of magnitude more. e efficiency and performance of an application are determined by the algorithm and hardware architecture, but critically also by the organization of computation and data. For algorithms with the same complexity—even the exact same set of arithmetic operations and data—executing on the same hardware, the order and granularity of execution and placement of data can easily change performance by an order of magnitude because of locality and parallelism. Especially in multi-stage algorithms like image processing pipelines, this difference comes not from the optimization of individual stages in isolation, but from the global interleaving of computations and data. For example, computing each stage completely before the next—even with the optimal inner loop, spread over thousands of threads on a GPU—destroys producer-consumer locality, constantly pushing data to and frommain memory. As a result, libraries of even the best optimized subroutines do not compose into efficient pipelines. I believe the key to turning ongoing exponential growth in hardware into orders of magnitude more performance and efficiency in real applications is to apply domain knowledge tomodel and understand the dependencies both in data structures and in the tasks linking them. e dependencies between computations and data de ne the topology of the problem. In graphics and imaging applications, the scale of both tasks and data are large, and the dependencies among them complex, making locality and parallelism essential but challenging to exploit. Modeling and understanding the dependencies allows us to: